Papers by Aida Mostafazadeh Davani
Distinguishing Address vs. Reference Mentions of Personal Names in Text (2023.findings-acl)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a core task in the NLP community . but not much work has been done to distinguish between addressing and referring to entities . |
| Approach: | They propose an automatic tagger that captures the address vs. reference distinction in English . they demonstrate how this distinction is important in NLP and computational social science applications . |
| Outcome: | The proposed tagger performs at 85% accuracy in distinguishing between address and reference in English . many modern Indo-European languages do not have such vocative case markers . |
GRASP: A Disagreement Analysis Framework to Assess Group Associations in Perspectives (2024.naacl-long)
Copied to clipboard
Vinodkumar Prabhakaran, Christopher Homan, Lora Aroyo, Aida Mostafazadeh Davani, Alicia Parrish, Alex Taylor, Mark Diaz, Ding Wang, Gregory Serapio-García
| Challenge: | Recent work shows that ignoring rater subjectivity is problematic within specific tasks and for specific subgroups. |
| Approach: | They propose a disagreement analysis framework to measure group association in perspectives among different rater subgroups. |
| Outcome: | The proposed framework reveals specific rater groups that have significantly different perspectives than others on certain tasks and helps identify demographic axes that are crucial to consider in specific task contexts. |
SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Models (2023.acl-long)
Copied to clipboard
Akshita Jha, Aida Mostafazadeh Davani, Chandan K Reddy, Shachi Dave, Vinodkumar Prabhakaran, Sunipa Dev
| Challenge: | Existing datasets on social stereotypes are limited in size and coverage . existing datasets are restricted to stereotypes prevalent in the Western society . |
| Approach: | They propose a broad-coverage stereotype dataset using generative models and a globally diverse rater pool to validate the prevalence of stereotypes in society. |
| Outcome: | The dataset validates the prevalence of stereotypes in society across 8 geo-political regions across 6 continents and states within the US and India. |
D3CODE: Disentangling Disagreements in Data across Cultures on Offensiveness Detection and Evaluation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies on annotator subjectivity focus on Western contexts and only document differences across age, gender, or racial groups. |
| Approach: | They propose a large-scale cross-cultural dataset of parallel annotations for offensive language in over 4.5K English sentences annotated by a pool of more than 4k annotators from 21 countries. |
| Outcome: | The proposed dataset captures annotators’ moral values along six moral foundations: care, equality, proportionality, authority, loyalty, and purity. |
On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning (2021.naacl-main)
Copied to clipboard
| Challenge: | PTLMs can exhibit biases against protected groups in a host of modeling tasks . but, fine-tuned LMs may propagate bias to downstream classifiers . |
| Approach: | They propose to use upstream bias mitigation techniques to reduce bias on downstream tasks by fine-tuning an upstream model and applying it to a downstream model. |
| Outcome: | The proposed model reduces bias on hate speech detection, toxicity detection and coreference resolution tasks over bias factors. |
Contextualizing Hate Speech Classifiers with Post-hoc Explanation (2020.acl-main)
Copied to clipboard
| Challenge: | Modern text classifiers struggle to learn a model of hate speech that generalizes to real-world applications. |
| Approach: | They propose a method to regularize BERT classifiers to detect bias towards identity terms by providing explanations for group identifiers and allowing models to learn from the context of group identifiers. |
| Outcome: | The proposed method limiting false positives on out-of-domain data while maintaining and improving in-domain performance. |
A Comprehensive Framework to Operationalize Social Stereotypes for Responsible AI Evaluations (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent years have seen unprecedented gains in generative AI models' capabilities across modalitieslanguage, image, audio, and video domains across the globe. |
| Approach: | They propose a framework to operationalize stereotypes in generative AI evaluations using social psychological research and NLP data. |
| Outcome: | The proposed framework identifies key components of stereotypes that are crucial in AI evaluation, including the target group, associated attribute, relationship characteristics, perceiving group, and context. |
Reporting the Unreported: Event Extraction for Analyzing the Local Representation of Hate Crimes (D19-1)
Copied to clipboard
Aida Mostafazadeh Davani, Leigh Yeh, Mohammad Atari, Brendan Kennedy, Gwenyth Portillo Wightman, Elaine Gonzalez, Natalie Delong, Rhea Bhatia, Arineh Mirinjian, Xiang Ren, Morteza Dehghani
| Challenge: | Existing estimates of hate crimes in the US are under-reported relative to actual number of incidents. |
| Approach: | They propose to use event extraction and multi-instance learning to predict hate crimes in local news articles for cities without official FBI reports. |
| Outcome: | The proposed model compares to FBI reports and shows that hate crimes are under-reported in local press. |
Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations (2022.tacl-1)
Copied to clipboard
| Challenge: | Annotators may systematically disagree with one another, reflecting their individual biases and values, especially in the case of subjective tasks such as detecting affect, aggression, and hate speech. |
| Approach: | They propose to combine multi-annotator models with multi-task based approaches to resolve disagreements between annotations and derive single ground truth labels. |
| Outcome: | The proposed model outperforms majority voting and averaging methods and estimates uncertainty in predictions. |